Many semiconductors can be used for acquiring a signal indicative of an image. Charge coupled devices (CCDs), photodiode arrays, charge injection devices and hybrid focal plane arrays are some of the more commonly used devices. CCDs are often used, since they represent a mature technology, are capable of large formats and very small pixel size and they facilitate noise-reduced charge domain processing techniques such as binning and time delay integration.
However, CCD imagers suffer from a number of drawbacks. For example, the signal fidelity of a CCD decreases as the charge transfer efficiency is raised to the power of the number of stages. Since CCDs use many stages, the CCD fabrication technique needs to be optimized for very efficient charge transfer efficiency. CCDs are also susceptible to radiation damage, require good light shielding to avoid smear and have high power dissipation for large arrays.
The specialized CCD semiconductor fabrication process is intended to maximize the charge transfer efficiency of the CCD. This specialized CCD process, however, has been incompatible with the complementary metal oxide semiconductor ("CMOS") processing which has been conventionally used. The image signal processing electronics required for the imager are often fabricated in CMOS. Accordingly, it has been difficult to integrate on-chip signal processing electronics in a CCD imager, because of the incompatibility of the processing techniques. Because of this problem, the signal processing electronics has often been carried out off-chip.
Typically, each column of CCD pixels is transferred to a corresponding cell of a serial output register, whose output is amplified by a single on-chip amplifier (e.g., a source follower transistor) before being processed in off-chip signal processing electronics. This architecture limits the read-out frame rate which the on-chip amplifier can handle proportional to the number of charge packets divided by the number of pixels in the imager.
The other types of imager devices have problems as well. Photodiode arrays exhibit high kTC noise. This makes it impractical to reset a diode or capacitor node to the same initial voltage at the beginning of each integration period. Photodiode arrays also suffer from lag. Charge injection devices also have high noise.
Hybrid focal plane arrays exhibit less noise but are prohibitively expensive for many applications and have relatively small array sizes.
In view of the inventors recognition of the above problems, it is one object of the present invention to provide an imager device which has the low kTC noise level of a CCD without the associated CMOS incompatibility and other above-described problems.
Many imaging applications, including biological vision modeling, stereo range finding, pattern recognition, target tracking, and progressive transmission of compressed images have made use of varying resolution image data. The availability of this data allows the user to obtain a unit, e.g., a frame of data at the lowest resolution necessary for the current task. This may eliminate unnecessary processing steps associated with obtaining a more detailed image. In the past, such multiresolution image data has been generated through an image pyramid approach. The observed scene is imaged at the highest resolution possible for the imager that is used. The next groups of pixel outputs are processed to create a combined output representing a lower resolution frame/image. This lower resolution frame/image is also stored. The process continues until a predetermined number of different resolution levels are obtained. The desired resolution level is then read out.
Many previous attempts used software to rearrange the image content. However, construction of the multiresolution pyramid through software can be a very computationally-intensive and time consuming portion of an image processing task. Many computers will consume on the order of hundreds of milliseconds for a 512.times.512 pixel array. This requires each resolution level to be individually processed and separately stored. The resulting processing time can make implementation of multiresolution readout in systems where data is required at video rates (e.g., 30 frames per second) difficult. The problem becomes even more severe for image processing tasks performed on large format images (e.g., 1024.times.1024 pixel arrays) where the output of millions of pixels can be involved.
It is another object of the present invention to provide a multiresolution readout system that can provide image data at a desired resolution and at speeds greater than or equal to the aforementioned video rates.
In view of the above, one aspect of the present invention is embodied in an imaging device formed as a monolithic complementary metal oxide semiconductor integrated circuit in an industry standard complementary metal oxide semiconductor process. The integrated circuit includes a focal plane array of pixel cells, each one of the cells including a photogate overlying the substrate for accumulating photo-generated charge in an underlying portion of the substrate and a charge coupled device section formed on the substrate adjacent the photogate having a sensing node and at least one charge coupled device stage for transferring charge from the underlying portion of the substrate to the sensing node.
In a preferred embodiment, the sensing node of the charge coupled device section includes a floating diffusion, and the charge coupled device stage includes a transfer gate overlying the substrate between the floating diffusion and the photogate. This preferred embodiment can further include apparatus for periodically resetting a potential of the sensing node to a predetermined potential, including a drain diffusion connected to a drain bias voltage and a reset gate between the floating diffusion and the drain diffusion, the reset gate connected to a reset control signal.
The imaging device also includes a readout circuit having at least an output transistor. Preferably, the output transistor is a field effect source follower output transistor formed in each one of the pixel cells, the floating diffusion being connected to its gate. Also, the readout circuit can further include a field effect load transistor connected to the source follower output transistor, and preferably a correlated double sampling circuit having an input node connected between the source follower output transistor and load transistor. The focal array of cells is also preferably organized by rows and columns, and the readout circuit has plural load transistors and plural correlated double sampling circuits. In this case, each cell in each column of cells is connected to a single common load transistor and a single common correlated double sampling circuit. These common load transistors and correlated double sampling circuits are disposed at the bottom of the respective columns of cells to which they are connected.
In the preferred implementation, charge is first accumulated under the image acquisition element of a pixel cell. That image acquisition element can be a photogate of an active pixel cell, an active photodiode, either of which can operate in either current or voltage mode. For simplicity, the image acquisition element will be referred to herein as a photogate, and with reference to a photogate. It should be understood that use of a photodiode will have certain differences from the photogate operation. The photodiode requires to transfer gate or photogate. The floating diffusion is therefore correspondingly larger. Also, the information is read out in reverse order--charge first, then reset level. This double sampling operation is therefore not correlated, and hence does not compensate for kTC noise.
Returning to the description of the preferred photogate mode, the correlated double sampling circuit samples the floating diffusion immediately after it has been reset, at one capacitor. The accumulated charge is then transferred to the floating diffusion and the sampling process is repeated with the result stored at another capacitor. The difference between the two capacitors is the signal output. In accordance with a further refinement, this difference is corrected for fixed pattern noise by subtracting from it another difference sensed between the two capacitors while they are temporarily shorted.
The imaging device can also have a micro-lens layer overlying the substrate. This micro-lens layer includes a refractive layer and individual lenses formed in the layer which are in registration with individual ones of the cells. Each of the individual lenses has a curvature for focusing light toward a photosensitive portion of the respective cell.
The aforementioned pixel cell structure can also be advantageously modified to allow the simultaneous integration of the array, thereby providing a "snapshot" image. The modification entails adding a storage gate which overlies the substrate and is capable of storing the accumulated photo-generated charge in an adjacent underlying portion of the substrate. In addition, the charge coupled device section has an additional charge coupled device stage which in combination with the original stage transfers charge from the portion of the substrate underlying the photogate to said sensing node. This additional stage includes an intermediate transfer gate disposed between the photogate and the storage gate. In operation, charge accumulated under the photogate is transferred to the portion of the substrate under the storage gate via the action of the intermediate transfer gate. The charge is then transferred to the floating diffusion via the action of the transfer gate only during readout. In this way, the entire array (or part thereof) can be integrated simultaneously and the accumulated charge stored until it is to be read out.
Any of the above-described arrays, or a photodiode array, can be also modified to incorporate a multiresolution imaging capability. This has been done by using a multiresolution circuit which is connected to each of the pixel cells. The multiresolution circuit processes the image signal output from each one of a group of cells forming a contiguous block within the array. The new blocks within the array form the new pixels of the new resolution image.
More specifically, the multiresolution circuit preferably averages the image signals output from the cells forming a block, to produce a block average output. This block average output represents a lower resolution image signal.
An important advantage of a block averaging process which produces a lower resolution image is reduced processing time. Noise reduction techniques are also possible. A block averaging process can be performed by the multiresolution circuit on the signals output by the pixel cell just after being reset, as well as on the signals output after the photo-generated charge has been accumulated and transferred. The two block averages, i.e., the block reset average and the block signal average, are then differentially compared to produce a reduced noise output signal. The multiresolution circuit can also be configured to average the aforementioned block averages over time. In this case, the image signal output by the pixel cells uses a succession of discrete image readouts, where each readout represents the scene viewed by the image device at a different time. The block averages produced following each readout are averaged for a prescribed number of iterations to produce the temporal block average.
The multiresolution circuit described above could also be used in an array where the actively photosensitive portion of each pixel cell is a photodiode.
An alternate version of the multiresolution circuitry is used to interconnect groups of the photodiodes forming an array to form contiguous blocks within the array, and output the block averages. Additionally, if such an array is used, it could be further modified to facilitate low light imaging. Specifically, the modified photodiode array would have pixel cells which include a photodiode, a floating diffusion, a readout circuit means connected to the floating diffusion, and a transfer gate between said floating diffusion and the photodiode. The multiresolution circuit would still be used to form blocks. However, the block would be read out via the floating diffusion of just one of the cells within a block. This enhances the output signal and allows low light imaging.
In addition to the previously-described benefits, other objectives and advantages of the present invention will become apparent from the detailed description which follows hereinafter when taken in conjunction with the drawing figures.